The instant invention comprises a related class of methods, techniques and system designs for improving at least the perceived, if not the actual, efficiency of and end-user""s interactive access of large (perhaps cross-referenced, cross-linked or xe2x80x9chyper-linkedxe2x80x9d) databases, such as the World Wide Web, via a communications network, such as xe2x80x9cThe Internetxe2x80x9d; and which can also be applied to the computerized access of local databases.
This document, to some extent, builds on inventive work disclosed by Inventor in earlier patent applications, issued patents, and disclosure documents, as well as during proprietary development. Where that earlier material is relevant it will, generally, be pointed out in the disclosure.
Generally, the domain of the instant invention is an extant (generally packet-switching) communications network such as xe2x80x9cThe Internetxe2x80x9d. Alternately, the invention can be adapted, in a straight-forward manner, and applied to a locally accessible database. Such a network comprises a multiplicity of computers connected via a network over which information is sent: from a first program running on a first computer to a second program running on a second computer; by (if sufficiently large) breaking that information into a multiplicity of packets; and, xe2x80x9cwrappingxe2x80x9d each packet in addressing information that specifies at least its destination. The specification of how information is parsed, addressed, transmitted, received and reassembled are generally called communication protocols and include, on the internet, TCP/IP, UDP, FTP and HTTP; as well as protocols that permit an end user to dial-up a service provider""s computer on the internet including SLIP, CSLIP and PPP.
The instant invention relates to a group of related methods for selecting and arranging information prior to its being submitted for distribution via the network and/or re-assembling the information after its being received at the destination. The network and its communications protocols operate on one or more levels below the operation of the instant invention and, thus, the instant invention may be considered a xe2x80x9cmeta-protocolxe2x80x9d or higher level protocol.
The intended practitioner of the present invention is someone who is skilled in designing such networks and implementing the software that affects and/or utilizes these communication protocols within the environment of a particular operating system (e.g., UNIX). That is, one skilled in the art of this invention is capable of writing program code and/or utilizing and coordinating existing programs within a multi-computer environment. Examples of such related tasks would be the programming of software to implement: a communications protocol such as TCP/IP, UDP, FTP, HTTP, SLIP, CSLIP or PPP; or, an application that utilizes such a protocol, such as an information providing server (e.g., an EMail distribution program, or an FTP or World Wide Web site) or information requesting and receiving client program (e.g., a user EMail program or Web Browser). Many of these capabilities or programs are provided as part of operating systems (e.g., UNIX) and such practitioners are knowledgeable about and skilled in implementing the internals of such operating systems. Often they are referred to as system programmers or software engineers.
Those skilled in the art know how to accomplish such tasks as to: coordinate and pass information between programs operating within a single computer or on several interconnected computers; initiate requests for local or remote information or database searches; and, generally, to accomplish these tasks by utilizing services provided by operating systems, communications packages and protocols, and other application programs. The details of accomplishing such standard tasks are well known and within the ken of those skilled in the art; are not (in and of themselves) within the scope of the instant invention; and, will be referred to but not described in detail in the instant disclosure.1 
Rather, what will be disclosed are novel higher-level methods, system designs and information arrangements, which rely and build upon those known and understood processes. Disclosed will be what one skilled in the art will need to know, beyond that with which he is already familiar, in order to implement the instant invention.
The above discussion relates to the instant invention in general. However, certain aspects of the instant invention comprise elements including database searching, statistical simulation or artificial intelligence (e.g., learning software and neural networks). Similarly, those skilled in these specialized areas of computer science are well versed in the details of those fields; and, generally, those details are not the substance of this invention. Rather, the disclosure will point out the specific elements of the instant invention to which the standard practices of those fields are to be applied; or, specifically how the standard practices are to be modified to practice the instant invention.2 
In summary, the disclosure of the instant invention will focus on what is new and novel and will point out, but not repeat the details of, what is relevant of that which known in the art.
This invention is disclosed in an technological environment that already includes, without limitation, large packet-switching communication networks, such as xe2x80x9cThe Internetxe2x80x9d, and large interconnected databases, such as the World Wide Web (WWW). Much of the disclosure will be cast as relating specifically to accessing the WWW on the internet; however, references made to xe2x80x9cThe Internetxe2x80x9d or xe2x80x9cinternetxe2x80x9d may be considered to apply to any similar data network (as well as to the process of accessing a local database) and references made to the WWW may be considered to apply, generally, to any type of local or remote, distributed or coherent, database. The WWW has been chosen as an example because of its complexity as a hypermedia database (which comprises text, images, sound, etc.) which is interconnected and distributed in nature; and, also, because of the current interest in, and increasing usage of, this particular database.
In addition there are many extant programs and mechanisms for accessing and initiating computer mediated searches of such databases. For example, gopher, WAIS, archie, veronica, web browsers (including Mosaic and Netscape).
Communication between various computers on the internet (and computer programs running on those computers) is affected by various communication protocols, including TCP/IP, FTP, UDP, HTTP, SLIP, CSLIP, PPP, etc.
There are already extant many programs and techniques for: running computer simulations and analyses of systems such as communication networks; compiling and analyzing statistics, including sensitivity analysis; and artificial intelligence techniques such as complex searching algorithms, programs that learn or train, can recognize patterns, neural networks, etc.
Lastly, there are a large number of service providers who will make available to users, who have personal computers or workstations, a connection via modem over a dial-up telephone line (or ISDN or dedicated line) through their computer to the internet. However, accessing, searching and browsing the WWW through such a provider can be a painfully slow process.
As explained, above, the process of xe2x80x9csurfing the Webxe2x80x9d, let alone trying to accomplish serious work, via the internet can often be a slow and frustrating process; especially if accessing via a modem through a service provider but, often, even if directly connected to the internet. Periods of being flooded with information may alternate with long stretches of waiting for information to arrive.
As working, playing or just browsing over communications networks, via digital computers, become ubiquitous, the number of users, number of xe2x80x9csitesxe2x80x9d, and the shear bulk of data being accessed, is growing exponentially.
The introduction of graphical or hypertext document systems (such as the HTML language, the HTTP protocol and browsers such as Mosaic) to the Internet, and the transformation of that network into the World Wide Web (WWW), have tremendously increased the requirements for bandwidth: both because of the increased requirements of graphic (or even video and audio) components of documents, xe2x80x9cpagesxe2x80x9d and files; and, because this richer, multimedia content is attracting users beyond a threshold where the commercial potential becomes obvious. This, in turn, prompts more xe2x80x9cinformation providersxe2x80x9d to take the plunge, increasing the size of the browsable pool of content, and a positive-feedback spiral results.
The increasing demands for raw bandwidth and xe2x80x9cnetwork infrastructurexe2x80x9d are creating a technical and logistical problem which is, on the other hand, perhaps an economic and business boon. The class of inventions described herein will likely only exacerbate this situation, as will become clear later.
Rather than being directed toward reducing requirements for bandwidthxe2x80x94i.e. traditional bandwidth reductionxe2x80x94these techniques are designed to increase the performance, efficiency and value of the work that can be done using whatever level of such communications technology is available. That is, the goal here is not to reduce the size of the channel needed, but toxe2x80x94apparentlyxe2x80x94squeeze more information down the available user channel.
This is similar to an approach used in Inventor""s invention which is subject of U.S. patent application Ser. No. 07/077,916 where Inventor states:
The goal of most bandwidth compression schemes is to xe2x80x9csqueezexe2x80x9d a given information signal into a smaller channel. A main goal of the instant invention is to xe2x80x9csqueezexe2x80x9d a larger picture into a given channel. Although this may at first sound as a semantic difference only, there is a significant practical difference.
Broadcast television in the U.S. is a xe2x80x9cgiven channelxe2x80x9d that is unlikely to change for the foreseeable future. Bandwidth compression schemes that would allow the use of a smaller transmission channel and require complex decoding at destination could not be used with the many millions of existing home receivers.
Similarly, most users are tethered to communications networks via telephone dial-up lines and modems whichxe2x80x94although, in general, modems get better and higher-bandwidth telephone service is becoming availablexe2x80x94for any given user, at any given time, are limited and usually not sufficient for the user""s desires. That is, working over a communications network is usually perceived as slower than working locally (or over a high bandwidth network, such as a Local Area Network or ISDN or T1 digital telephone line).
However, whatever the given network connection, these techniques can be used to advantage to increase actual or perceived performance (of work, if not strictly of bandwidth) for many types of work done over such networks.
In particular, searching or browsing through large amounts of information (cohesive databases, distributed masses of information such as the WWW, or otherwise) is a task that often goes in fits and starts. The user makes some decision about what is desired and makes a request for more remote information: as a formal database search request; by clicking on a xe2x80x9clinkxe2x80x9d on a WWW page; or otherwise. The user then waits for the information to be delivered to his local screen. During this period the user often sits idle. The user then reviews the new information. During this period the local computer and communication line often sits idle. And so on.
The goal here, then, is to: speed up the overall process and make it more efficient; make the user feel that the process is more immediate and valuable; shorten work time; or, alternately, get more done in the same time.
In an early stage of the development of the techniques described herein, Inventor recognized that: remote computer databases that are searched for professional purposes are becoming more widely used, particularly in the legal, engineering/scientific and medical fields. Further, the amount of data to be down-loaded is increasingly large.
Communications and on-line costs are now often a significant part of the cost of doing business. Efficient use of communication and database search facilities is therefore desired.
We have developed a group of related techniques (technical details are described below) that will make more efficient use of the open communication line, saving money. These will also speed the search procedure, saving the working time of the lawyer or other professional using the system, which professional time is much more valuable than communications time.
These techniques will also be useful for local searching of databases contained on magnetic disk or CD-ROMs.
A narrative example is as follows: A lawyer has requested from a remote database, and is reviewing on screen, a particular legal case. In the most simple embodiment, all cases referred to by said case would be retrieved by the remote computer and initial sections (pages or paragraphs for example) of each case would be transmitted to the lawyer""s workstation. In this way when the lawyer was ready for a next case, it is highly likely that some of that material would already be available at the lawyer""s station for review. As soon as a selection is made by the lawyer, transmission time is wholly devoted to that selection rather than multiplexed between multiple potential selections. The relative amounts transmitted of each potential selection, and the list of potential selections, can both be modified by factors such as: the length of each document; statistical xe2x80x9cknowledgexe2x80x9d of the lawyer""s prior access to that particular document; statistical xe2x80x9cknowledgexe2x80x9d of the lawyer""s prior access to that type of document; document dates; search of abstracts and key words; prior statistical knowledge of lawyer""s work style; and other factors.
As explained, above, communication while accessing remote databases over the internet often runs in fits and starts; the communication channel is often left fallow. [At least the communication line between a service provider and an end user; the communication line between the service provider and the internet, in general, may be in more uniform use, serving many users simultaneously.] The intent of the instant invention is to make use of that fallow time and, thereby, improve the perceived (if not actual) responsiveness and efficiency of the network to the user""s demands.
A specific deficiency that arises when accessing WWW xe2x80x9cpagesxe2x80x9d is that each page consists of a text script and, optionally, a number of associated image or other multimedia resource files. A separate communication connection (a logical data construct, not a physical wired connection) is established, used, and then broken down for the delivery of the script and each resource. This use of a multiplicity of connections results in a significant amount of overhead and delay. One element of the present invention will be to establish and leave extant a communication connection that will survive the delivery of a single file, and which can be used not only for delivery of all the resource files comprising a document or xe2x80x9cpagexe2x80x9d, but for the delivery of multiple documents from the same source to the same destination. This technique, in and of itself, may not constitute inventive matter; however, the composite method, which this and related techniques comprise, and new uses to which they are put, are.
Similarly, it is standard that an end user""s client program (that program which permits the user to xe2x80x9cbrowsexe2x80x9d and which initiates system requests for information resources) permits a single request to be made, waits for the requested information to be delivered, and then waits further while the user reviews the delivered item and decides what to do next. Only then is the user permitted to make a next request. In the instant invention multiple, parallel requests will not only be permitted, but in some embodiments the system will make those multiple requests in anticipation of user requirements. Again, this technique, in and of itself, may not constitute inventive matter; however, the composite method, which this and related techniques comprise, and new uses to which they are put, are.
The more substantive elements of the present invention comprise the following:
1. The interleaving or time-division multiplexing of multiple data resources (generally, files) into a single (or at least composite) communication stream of information, by fragmenting the resources into portions and interleaving those portions.
2. The weighting of the presence of the individual data resources within the composite information stream by modulating the size and/or frequency of representative portions to be included.
3. Attempting to anticipate the user""s likely future requirements on a xe2x80x9cpredictive probabilisticxe2x80x9d basis. Further, the flexible and/or artificially intelligent adjustment or customization of the algorithms that predict and determine the interleaving weights for a particular user.
4. Determining and updating the interleaving weights (2) based on the anticipation (3) and actual events.
5. The xe2x80x9cpyramidizationxe2x80x9d of data resources in such a way that a partially delivered data resource is, at least, of some use to the end user; perhaps, if only, to evaluate whether the balance of the data resource is required.
These five elements will be disclosed in more detail below, as will their integration into a class of methods and protocols that will permit more efficient usage of communication networks by end users. In particular, new communication protocols and methods will be disclosed that function: as a layer above existing protocols; as a layer below existing protocols; as addons or extensions to existing protocols; or, as entirely new protocols or methods.
Further, it will be disclosed how to apply these new methods to a number of different system architectures. Thus, a group of related embodiments will be disclosed which can be appropriately applied to different situations. Not all of these variations will embody each of the above elements to the same degree, if at all.